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12 February 1975 


MEMORANDUM FOR: Lt. General Samuel V. Wilson, USA 

Deputy to the DCI for the 
Intelligence Community 

-t. 

SUBJECT : Devilish Advocacy 


1. I have carefully gone over your 5 February memo- 
randum to the DCI on "Challenge Procedures for the 
Intelligence Community," with its annexes, and have 
discussed this topic with my colleagues. Our thoughts 
and comments are outlined below. 

2. No one feels more acutely than my NIG colleagues 
and I the absolute necessity for ensuring that intelligence 
appreciations and Estimates (small e or capital E) sent 

to the President and NSC- level consumers reflect the best 
evidence and judgments available to the entire U.S. 
Intelligence Community. To us, this entails ensuring that 
divergent views on matters of substantive importance are 
accurately reflected in a manner that enables the reader 
(consumer) to assess their force and the evidence and 
reasoning on which they are based -- i.e., in a length 
and a scope which permits an advocacy presentation, right 
in the body of the paper and not in an agate-type footnote. 
Also, we are well aware of the inherent danger and high 
risk of error involved in two types of Estimates: those 
in which knowledgeable opinion throughout the Community 
is virtually unanimous and those in which a mistaken 
Estimate (e.g., the Soviets will/will not put offensive 
missiles into Cuba) could have an adverse impact on U.S. 
interests by contributing to the wrong policy decisions 
on matters of major importance. Even given all the above, 
however, we still feel it would be an error to institu- 
tionalize the devil’s advocacy procedure. 

3. With specific reference to your 5 February memo- 
randum, the first of your specific recommendations reflects 
what I believe to be and am actively working to ensure 

is the current situation. Consequently, I do not consider 
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it accurate to describe this as a proposal for needed 
change. 

4. Your remaining proposals (the balance of your 
memorandum's paragraph 3, appearing on its second page) 
involve the schematics of an institutional process which 
I consider not only unnecessary but ill-advised* 

5. The type of procedure we are discussing is not 
really applicable to all Estimates, only to some and the 
principle of differentiation has nothing to do with the 
subject matter, length or deadline pressure of the Esti- 
mate in question. On papers over which there are genuine, 
spontaneous differences of opinion among those participating 
in such a paper's production, no external devil's advocate 
is needed. Conflicting views will be argued, vigorously, 
out of genuine conviction -- not because one has been 
retained to take a brief. What is required here is simply 
that the NIO responsible for such a paper ensure that the 
chairman allow full scope to the interplay of debate. 

The whole NIO structure was deliberately set up to facili- 
tate this and encourage it. What you are asking here, 
therefore, is that the NIOs do the job the DCI has asked 
them to do. 

6. The papers on which some form of advocacy procedure 
might be desirable are those which may be on controversial 
subjects but happen to be ones where the views of knowledge- 
able experts within the Community have a high degree of 
overlap, i.e., where there is a majority opinion so pre- 
ponderant as to be virtually unanimous. Here we do have 

to be careful about the dangers of reinforcing consensus and 
the psychological difficulty of the lone dissenter bucking 
the majority tide, particularly when everyone is under 
pressure, an impossible deadline is fast approaching, nerves 
are frayed, and people are not particularly receptive to 
what they instinctively regard as idle argument. Even here, 
however, a prearranged advocacy institution does not seem 
to me to be the best answer to this sort of real and concret 
problem. 

7. One difficulty with the advocacy concept is that, 
plausible as it sounds in abstract statement, it rests on 
some assumptions which are at variance with the way 
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Estimates (again, small e or capital E) are in fact 
produced. Despite the skeptical contrary views of some 
of our consumers or even some of our colleagues outside 
the estimative process, the drafters of an Estimate do 
not start with conclusions and then begin casting about 
for evidence to support .them. Thus, it cannot be known 
until fairly late in the game what a key paper’s judgments 
will actually be or the extent to which there will be 
spontaneous disagreement over these judgments within the 
Intelligence Community. An officer or an office tasked 
with opposing anything a paper came up \'jith and being 
negative at all stages of its development would have a 
rather sterile job which would contribute relatively 
little to the quality of the end result. In most concrete 
cases -- not all admittedly, but most -- you will not know 
until near the end of the production process whether you 
have a reinforcing consensus problem, i.e., a situation 
in which a devil’s advocate might be useful. 

8. Institutionalizing the process also raises 
another potential difficulty, mentioned in your memo’s 
annexes. If all or virtually all of the Community's 
knowledgeable experts in a given field are in unanimous 
agreement on a key judgment, that fact alone ought to 
tell the policy- level consumer something. Majorities 

are sometimes wrong, but it is rash to make the assumption 
that they are invariably wrong and such an assumption 
indicates a rather disquieting lack of confidence in the 
professionalism (including objectivity) or knowledge of 
the Community's analysts. It would be mischievous or 
misleading to include a devil's advocate annex to an 
agreed paper in a way that would permit a consumer to 
infer that the annex had behind it the same weight of 
knowledge and analysis as the paper itself. 

9. I do agree, as do my colleagues, that we have to 
be particularly careful in papers where there is an almost 
unanimous consensus or ones whose judgments have a direct 
bearing on vital U.S. interests along the lines indicated 
above -- i.e., ones in which judgmental errors could be 
disastrous. In such cases the NIO should do the follo^^ring 
things : 


a. Deliberately pause to focus on arguments or 
conclusions that run counter to the paper’s main 
judgments. This could indeed in some instances 
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involve picking one of those who had participated in 
the paper’s preparation or some knowledgeable analyst 
who- had not actually been part of the process and 
asking him to give a forceful summary of the opposi- 
tion case. Utilizing this technique on the occasions 
where it is concretely advisable, however, is not 
the same thing at all as institutionalizing it or 
making use of it as an elaborate mechanism ’which 
could easily degenerate into something pro forma. 

b. On those Estimates where judgmental errors 
would be serious, adding an annex stating that the 
drafters and producers of the Estimate are aware of 
the consequences and dangers of error and have not 
lightly arrived at their judgments. Such an annex 
could well enumerate the main contrary arguments 
and explain (succinctly) why those participating in 
the paper’s preparation had been led to reject them 
after giving them careful consideration. 

c. In certain areas where there are consensus 
views not related. to a particular paper but which 
are likely to. shape all papers written on that 
subject, commission a special internal study to 
fish-eye those judgments. A case in point here is 
the virtually unanimous opinion, reflected in the 
latest NIAM, that in any renewed Arab- Israeli 
conflict (barring Soviet intervention) the Israelis 
would soon emerge victorious on the field of battle. 

A hard look at this judgment is, in fact, already 
underway at my request. 

. 10. In sum, the problem which led to General Graham’s 
original suggestion and has stimulated the DCI’s concerns 
is a real one. My colleagues and I are as aware of and 
bothered by it as anyone else in the Community. It needs 
address and we are endeavoring to address it. An elaborate, 
formal devil's advocate mechanism, however, does not seem 
to us to be the optimum way to tackle it. 
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5 February 1975 


MEMORANDUM FOR* Director o£ Central Intelligence 

\ 

SUBJECT* Challenge Procedure* for the 

Intelligence Community 


1. Some time ago you expressed interest in establishing 
within the community a regular system for the presentation of 
the views of devils* advocates, i* a. , some sort of system which 
would ensure that majority views and the conventional wisdom 
concerning major intelligence judgments would be subject to 
effective challenge procedures* Though few would quarrel with 
the objectives of this proposal, several have, in fact, questioned 
the practicality of institutionalizing challenge procedures and have 
expressed the fear that the system's (nan-monetary) costs might 
outweigh its benefits. 

2. ’ My own feeling about the subject are mixed* I very 

much favor the concept of regular challenge procedures, 
particularly as a part of the normal production process, but 
recognize that there will be pitfalls attending their establishment 
as a separate institution. Accordingly, I recommend a course 
which in genera l-thought not in all its specifics* -follows the 
route laid out H ~j n the paper at Tab A), but which 

also seeks to avoid some of the dangers identified bj 

(in TabB). 

3* Specifically, what I propose can be summarized as 
follows* 


25X1 


-- The MO responsible for a given paper will in all 
instances ensure that minority points of view and dissents 
are adequately represented and discussed. He will, in 
addition, report to USIB (in the case of NIEs and NlAMs) 
or to the DCI (in the case of other major papers) concerning 
principal issues in dispute, if any, and the extent to which 
he and his committee pondered contrary opinions and 
judgments. 
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— When appropriate, the D/DCI/NIO will appoint a 
devil 1 s advocate (DA) to represent dissenting views* An 
official DA will normally be appointed when an interagency 
paper {1) is considered to b© of unusual significance to US 
interests and policies; (2) contains judgments which, axe 
clearly controversial; or (3) makes estimates which, if ; 

wrong, would likely have very important (and adverse) \ 

effects on US attitude 9 ^and policies* f 

v A 

*•' ■ 

-- The DA, a senior and escoeriencad officer in. the \ 

h. ; 

commuxiity, will b* responsible for formulating and 
representing dissenting views throughout the life of the; • 

paper under consideration, i. e, , from his initial appoint- l 
meat (which could occur at any time in the production t 

procesa) through USIB discussion. 

— The DA will be responsible for soliciting the views 
ox other dissenters within the Community. He will also \ 

provide the DCX or USIB with an account of his service . \ 

before the given paper is approved for publication. 

-- The NIO and the DA will, in those instances when 
mistaken estimates might result in very serious damage to 
US interests, provide the DCI or USIB with a written statement 
concerning the possible implications of misestimating. 

-- On those occasions when a paper is time -sensitive 
it will be incumbent upon the DA to accomplish his tasks 
with considerable dispatch. 

The NIO and DA will follow the procedures outlined 
in detail in Tab A from p. 5 through p. 10 (under the heading, 
"And Some Specific Proposals"). 

4. If you approve, we are prepared to circulate this proposal 
to USIB for its consideration at some future meeting. 


A ti a chm s nt 3: Asst at e d 
cct C/CCI/NIO w/att 


Samuel V. Wilson 
Lieutenant General, USA 
Deputy to che DCI for the 
Intelligence Community 


n 
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MEMORANDUM 

SUBJECT: Challenge Procedures - A Proposal 


So me General Considerations 

The DCI first proposed the establishment of challenge 
procedures in October 1973, when, in the aftermath of the 
failure of intelligence to warn of the impending Arab attack 
on Israel, he and others suggested that a means be found to 
introduce some form of devil's advocacy into the Community s 
analytical proceedings."' 

Briefly, what seems to be needed is a challenge procedure 
which would, inter alia, assist production analysts to overcome 
three occupational hazards to which, according to our post-mortem 
reports, they are occasionally subject: 

Preconceptions: the tendency to discount information. 

that runs counter to long-held convictions; 


❖ Specifically, the DCI stated: "The IC Staff. . . will develop regular 
systems to be implemented by the NIOs to ensure that serious 
divergent points of view and conflicting elements of information 
not be submerged by managerial fiat or the mechanism of rein. 1.01 cing 

consensus Such systems will also be charged with ensuring the 

establishment of means to provide the views of devils' advocates, 
adversary procedures, and the use of gaming techniques as 
appropriate. " ("The Performance of the Intelligence Community 
Before the Arab-Israeli War of October 1973: A Preliminary Post- 
Mortem Report, " submitted by the PCI, December 1973, p. 22 . ) 
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--Reinforcing consensus: the tendency for divergent 
views of individual analysts to be submerged ift a sea of 
conventional collective wisdom; 

--The current intelligence syndrome: a myopic view 
of the forest because of forced focus on current intelligence 
trees. 

The notion that some way should be found to challenge conventional 
substantive judgments has by now itself become quite conventional. 

But perceiving the wisdom of establishing challenge procedures does 
not lead easily into an appreciation of precisely what form such 
procedures should take. It is the who , when , how , and why of 
challenge procedures which confront those who seek improvements 
in analytical performance: 

o just who in the community (and with what credentials) 

is entitled to challenge whom? 

o just when (and vis-a-vis what) should the challengers 

perform their art, i. e. , challenge? 

❖ Conventional enough, at any rate, to elicit challenges from those 
who see only problems in a devil ' ? advocate procedure. 

2 

l 
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© precisely how should the challenger perform, i.e. , 
through what system or medium should he present his contrary 
views ? ’ 

© and, finally, why should the, challenger risk his (and 
perhaps his sponsoring agency's) reputation for sagacity for 
the sake of representing, by definition, a singular and probably 
unpopular point of view? 

Some Guiding Principles 

Clearly, some general rules of the game should be developed. 

A challenger should possess substantial substantive competence in 
the area under exploration, but he should not be so close to the subject 
that he lacks perspective and suffers from the analytical disabilities 
listed above. He should be familiar with Community processes and, 
to a degree. Community "politics. " And he should be articulate and 
persuasive. (He need not believe his Own advocacy, but he should 
be devilish enough to convince others that he does. ) Finally, and 
obvioxisly, he should have the time to do the job properly. 

When should this paragon exercise his persuasive talents in this 
strange manner? Certainly not as a matter of routine concerning 
ordinary subjects. He should confine himself to matters of high 
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moment (those of long-term significance as well as of immediate 

* 

concern) and papers of particular consequence {NIEs, Alert Memo- 
randa, etc. ). And he should deal, full-time, only with specific 
circumstances, problems, and/or papers for a stated period (weeks 
or months). A more or less permanent assignment as a devil’s 
advocate would soon sap the vitality and demolish the credibility 
of even the most enthusiastic and skillful practitioner. And the 
devil probably would, over time, withdraw from the process. 

There already is, in being, Community machinery, the NXO 
system, which could accommodate - -effectively, if not easily- - 
the establishment of challenge procedures. No other component 
of the Community performs so many significant substantive functions 
for the DCI and USIB; no other component is so close to the consumers 
and no other component is so involved in the production of important 
Community assessments. And from the point of view of the advocate 
himself, no other component could so readily provide him with the 
papers, contacts, forums, and general support necessary for the 
practice of devils' advocacy. 

Indeed, in our canvass of alternatives, we could find no other 
office or mechanism which could properly support a regular devil’s 

4 
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advocate process, and we do not — for a variety of reasons --favor 
the establishment of a new institution to perform this task. 

And what of the risk of serving as a champion of unpopular 
causes? Perhaps the devil's advocate --assured that he would 
serve only a short tour, and guaranteed a respectful (if not 
sympathetic) audience --might find the exercise of his imagination 
and the influence he brought to bear on the weighty judgments of 
the Community reward enough. And, at the conclusion of his 
sentence, he might find some solace in the drink which the NIO 
will buy for him at Sans Souci. 

And Some Specific Proposals 

Assuming, then, that. challenge procedures are to be instituted 
within an NIO- sponsored system, we would propose the following 
specific measures: 

A. The DCI or the D/DCI/NIO, in consultation with 
US1B, if appropriate, should determine if a given subject and 
paper seem sufficiently important to warrant the institution of 
official challenge procedures. (A standard country paper on, say, 
Argentina would probably not so warrant; an SNIE on, say, Soviet 
reactions to specific US courses of action, almost certainly would. ) 


5 
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USIB principals (and perhaps other senior figures in the Community) 
should be encouraged to nominate candidates for devil's advocate 
positions. In consultation with the D/DCl/NIO, the D/DCI/IC, 
and others, as appropriate, the NIO responsible for a given paper 
should then appoint an intelligence officer as a substantive challenger, 
responsible as a devil's advocate for the effective presentation of 
contrary substantive views. 

1. The possible appointment of a challenger should 
be discussed early on (perhaps during Terms of Reference 
meetings) with the various agency representatives and with 


USIB. ~ 


2. The designated devil's advocate should be an 


intelligence officer who has appropriate substantive credentials, 
experience, and seniority. He could be drawn from any 
element of the Community and might, in some circumstances, 
be another NIO, perhaps one with related substantive responsibilities. 

3. The devil's advocate should play his role for the 
life of the paper concerned- -in the case of an NIE, from the TR 
stage through USIB consideration; in the case of an Alert Memo- 
randum, from its inception through the DCI's approval. He 
should, in effect, serve a temporary tour in the devil's advocate 

capacity. (> 
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4. There are of course practical obstacles to 

t 

such a procedure. Few if any components will feel free 
to spare a senior officer for devil's advocate duty. The DCI 
may thus wish to ask USIB principals to consider this 
procedure to be a necessary part of USIB's set of substantive 
and estimative responsibilities --and, in effect, as significant 
as providing, for example, representatives to attend meetings 
on NIEs. 

5. In some instances (e.g., NIE 11-3/8), the NIO 
might wish to consider the appointment of more than one 
devil’s advocate, depending on the size, complexity, and 
diversity of the paper under preparation. 

B. The challenger would be charged with: developing and 
presenting plausible arguments against the conventional wisdom 
and against any or all of the papers' major judgments; expressing 
disbelief or skepticism about certain specific pieces of evidence 
and/or discrete conclusions based on them; challenging the logic 
and coherence of given lines of argument; and identifying any gaps 
in information and "holes" in argumentation which he perceived 
as affecting major judgments., 
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1. The devil's advocate would be responsible for 

* 

eliciting the views of other responsible officers in the 
Community who held views contrary to those offered in 
the paper under review (contrary to those offered in 
expressions of dissent as well as in the main text). 

2. The devil's advocate should present his 
advocacy orally, during meetings on the paper, and, when 
appropriate, in writing. The NIO in charge would be duty- 
bound to respond to the challenger's case, though would be 
empowered to accept or reject his advice after due considera- 
tion. The degree to which the devil's advocate was able to 
influence substantive judgments in the paper would thus 
ultimately be determined by the NIO. 

3. All papers subjected to this form of challenge 
procedure would bear a notation to that effect somewhere 
(in an introduction, opening statement, footnote, whatever) 

in the paper. In some instances, the burden of the challenger's 
argument (especially if otherwise ignored) could also be 
succinctly stated. For example: . 

.8 
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The principal' conclusions of this estimate 
were subjected to a critical review by the Com- 
munity's "devil's advocate. " The contention of 
the advocate that, in the circumstances described 
in the paper, the Arab governments would almost 
certainly sue for peace (beca.use they would be 
principally concerned about their own survival) 
was carefully considered. This argument was, 
however, rejected, in the main because of reasons 
discussed in paragraphs 14-16 of the text. • 

4. Occasionally, however, the NIO might wish to 
reproduce the devil's advocate's case in extenso? If S o, that 
case should be interwoven with the main text. This would 
minimize repetition and confusion and place arguments in a 
proper context. (Most NIEs, etc. , already march in this 
manner, and should continue to do so, though the pros and 
cons of course should not be presented in a way likely to 
obscure the conclusions. ) 


Indeed, it has been suggested that many papers take the form of 
straight advocacy but provide a separate annex setting forth the 
paper s pros and the devil's advocate's con arguments; or that the 
main text itself should interweave these pros and cons. Adoption 
of the first suggestion— in effect the publication of a devil's annex- 
would, with perhaps very rare exceptions, probably only irritate 
and confuse the consumer. 
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5. Whatever the impact of his role on a given 
paper, the devil’s advocate should make his case available 
to other elements of the Community (e. g. , current intelligence 
components) so that they, too, could be informed by his 
advocacy. 

1 25X1 
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SUBJECT: A PROPOSAL FOR A "CHALLENGE MECHANISM” FOR 

THE INTELLIGENCE COMMUNITY 


Introductory Bote: This paper looks at the feasibility of 

a "challe-nge mechanism or "Devil's Advocate.," 
as it has been termedj in the Intelligence Community . The paper 
does not proceed with a full discussion of the pros and cons of ' 
formally jr^tiytutignalizi^ challenge . Bather , it seeks to explore ' 
the working milieu In which an institutionalised challenge mechanism 
would have to function . This should enable interested parties to come 
to some conclusions about the feasibility of the concept . Our method 
has been to solicit the views of individuals who formerly held, or 
hold now, key managerial positions in substantive intelligence-producing 
organizations , and IlIOs , and to let their views speak for themselves . 

We have concentrated on these individuals since we sought to ' emphasize 
feasibility of challenge in practice, and these folk are critical to 
the success or failure of the process. Finally, although sentiment 
of those reo.ched runs rather heavily against ins ±i tutiondli zinc ' 
challenge , a large number of possible ways to do just that are set 
out for possible consideration . - 

Jr * * jE 2 jE jE jE jE * * j£ 

I- WHY A CHALLENGE MECHANISM? ; 

The concept was first advanced by the D/DCI/IC in a 

memorandum for the DCI which may be found at TAB A. 

Briefly, in that memorandum, the notion was advanced that 
- > 

a "Devil's Advocate" would be useful in forcing a- close 
look at major judgments, and the possibility of institution- 
alizing that role should be examined. 

An unstated but clear implication of that proposal 
was that in the preparation of major substantive papers, 
such as NIEs and InterAgency Memoranda, at least some key 


draft 


Approved For Release 2004/05/13 : CIA-RDP91M00696R0005001 50005-1 



. Approved For Release 2004/05/13 : CIA-RDP91M00696E000500150005-1 

minority views were not being ventilated to the fullest 

extent, that other views were or could be overlooked, or 
that important contingencies might not receive full . attention 
In looking into that hypothesis, it was thought useful 
to compare production and particularly the. review process 
for estimative papers as it obtained in the former Office 
of National Estimates (ONE) and as 'it is now under the 
auspices of the National Intelligence Officers (NIOs) . 

Briefly, in O/NE there were several levels of reviews 
built into the normal production process: 

a) Drafts were initially reviewed by .the Staff 
Chief in a regional/functional staff. Some, but not all, 
regxonal/functional staffs held rather searching internal 
reviews of" drafts which' provided an opportunity for other 

m-staff views , before forwarding papers to the O/NE front 
office; 

b) Normally when sending the draft paper to O/NE 
front office-copies were sent to other offices in CIA with 
an interest in the paper requesting informal comments; 

c) The Board of National Estimates considered 
the draft. Normally, there were a variety of views on a 

paper, with one or more Board members acting in fact as a 


associated with challenge, but particularly that of 
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representatives from other CIA offices were included, and 
their views were heard; 

d) Next, representatives of the USIB agencies 
met; in addition, representatives of the various CIA offices 
of interest also attended these sessions as bach-benchers ; 

e) USIB considered the paper. 

There were two other devices used by 0/JME to offer the 
opportunity for dissenting views to be ventilated. Staff 
Memoranda were not infrequently produced by O/NE analysts 
which, in effect, said, "Yes, most of the Community thinks 
the situation in Ruritania is progressing along a certain 
path, but here are some very good reasons why it could go 
quite another." After consideration by the Board, ONE made 
a judicious distribution of these memoranda. 

Another device was to present key estimates to a 
prestigious group of consultants at tri-monthly meetings at 
Princeton. Consultant comments and a lively give— and-take 
ensued on some papers. Ambassador Komer's comment seems 
to be a generally -held reaction. "The old O/NE consultants 
were not very effective" (the knock is not against those 
particular consultants but rather on the difficulty of 
bringing outside consultants "up to speed" with respect to 

3 * . 


Approved For Release 2004/05/13 : CIA-RDP91M00696R000500150005-1 



given MRT6^f!i& 0 4S e l ease 2004/05/1 3: CIA-RDP91M006Q#R0005001 50005-1 

Under the present system, where the NIO subcontracts 
a paper for drafting to some element of a USXB agency, 
the levels of review are fewer. For example, if a draft 
is produced by an analyst in O/CI or DIA's Directorate for 
Estimates it goes directly to the NIO. To the degree the 
NIO reshapes it there is a review, and the NIO can, of 
course, act to challenge all or any portion of the paper. 
Also, the NIO can establish some kind of ad hoc group to 
review a paper. Infrequently, as in the case of NIE 11—15— 

74, a Committee of Experts' - looks at the draft. Normally, - 

* 

however, after his own review, the NIO would send the draft 
to the USIB representatives for their consideration (item d. 
under previous O/NE procedure) . In practice, therefore, 
the levels of review are numerically less than before, and 
to the extent one NIO can review a paper (vice a panel of 
senior officers as in O/NE) , given time and other pressures, 
the quality of review may not be as profound as. before. 

The concept of the "noninstitutional draft,” while 
perhaps healthy for other reasons, also probably inhibits 
the amount and degree of 'Devil's Advocating* that existed 
previously. Specifically, if the NIO levies a draft on 
DIA, and makes it clear that he seeks the drafter's views — 
not the so-called "DIA pitch" — he will get that drafter’s 
views, if DIA or OCI or whoever is drafter, plays the game. 


4 
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get the kind of quality control within PI A that he v/ould 

if DIA were producing a formal paper under, its byline. 

The NIO will get the best effort of the drafter, affected 
to some degree by the advice of his colleagues, but he 
will not get a paper which has undergone . searching review 
and questioning— DIA will reserve that for the USIB repre- 
sentatives meeting or, in extremis, for the USIB meeting 
itself. Certainly, the draft will not undergo a very close 
review comparable to papers moving through the O/NE 
production process. 

Another relevant factor arises as a result of the 
demise of O/NE drafting responsibilities, which might be 
termed the current intelligence syndrome.’'- Under the 
current arrangement the bulk of all papers prepared under 
NIO auspices must be produced by substantive organizations • 
geared in the main, toward producing current intelligence 
These include: CIA/OCI, State/INR, and DIA/DI tin these 
three organizations are found the bulk of the Community 
analysts capable of nroducing finished national intelligence 
drafts) . CIA/OSR and CIA/OER are to a somewhat lesser extent 
o much concerned with production for current publications. 
Only DIA/DE lacks current intelligence responsibilities 

among the several organizations on which the NIo’ must draw 
for drafting support. 
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heavily on current intelligence analysts to also prepare 

estimative and longer-range judgments, at least in the 

sense of reducing the opportunities for other views to 

impact on current x^isdom. This problem, i.e., the weight 

of the current intelligence apparatus, was adverted to 

in a memorandum prepared recently by a senior NIC analyst. 

She. was considering the warning function specifically, 

but her point may have a xxader validity. 

"*••-* The fact is that the current intelligence 
structure in each agency - ’ — with such assistance as 
it may require from other specialists such as order 
of battle analysts, economists or scientific experts — 
virtually has a monopoly on the daily and weekly 
intelligence production effort which flows to the 
intelligence chiefs, policy officials, military 
commanders, and the White House. Each day, the' 
agency and often the Community position is established 
on critical current issues as they are set forth in 
the daily publications and briefings . . . . • Not 
surprisingly, once these items have been disseminated, 
there is the strongest resistance in every agency 
to the dissemination of any differing interpretation, - 
X'/netner produced by warning analysts or anyone 
else. . . . . ” ■ . 

The question can be reasonably asked whether the 

current intelligence analyst is the best person to make 

the longer-range assessment, but even if he is, the larger 

problem may be that longer-range estimative and assessment 

. papers may be (perhaps unconsciously) reflecting current 

intelligence • wisdom. This vulnerability is enhanced by 
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being made is ’Tfiat levels of review, if properly handled, 

offer opportunities for adversary views to be exposed, 
not merely that levels of review are useful, per se) „ One 
NIO pointed out that "while under the NIG system schematically 
the opportunities (for reviev?) had been reduced, "that was 
not really the case." He pointed out that on CIA drafts 
he conducted an "in-house CIA review" prior to issuing the 
paper for the USIB representatives meeting- However, he 
also noted that this was not done in all cases by any means. 

Hence, on the grounds not only of the original 
suggestion by D/DCI/IC, but because several review -levels 
had been eliminated with the demise of O/NE, plus the 
possibility that current intelligence views might be so 
dominant in production that the content of longer-range 
papers might be affected, we decided to explore the working 
mileau in which an institutionalized challenge mechanism 
of some sort would have to function. 

II . VIEWS ON THE WORKING MILIEU FOR A CHALLENGE MECHANISM ■ 
The concept of institutionalizing a challenge mechanism 
aroused considerable skepticism and doubt as to its 
effectiveness and utility from nearly all those with, whom 
it was discussed, both current and former practitioners. 
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in experience and practical problems that face substantive 
intelligence producers and managers. There is no reason 
to believe that it is simply blind opposition, although 
practical bureaucratic considerations have some impact. - 
Before examining some of the reasons for doubt over 
institutionalizing dissent described by those reached in 
the survey, an important point must be made. That is, all 
believe very much in dissent, adversary procedures, devil's - 
advocacy and the like, as a necessary and integral part 
of the process of producing good intelligence. But virtually 

i 

all also are substantially in accord with the' view of a 
former DDI who puts it this way, dissenting views 

can most effectively be dealt with at the working level of ' 
review, .indeed as early as possible in the production 
process." A former Assistant Deputy of O/NB. contends, "A 
kind of Devil's Advocate should be part of the process in 
working up a paper through the working substantive levels . . . 
it is all part of the 'tightening process' in producing 
any paper." A current manager says, "the way to achieve 
this (introducing alternatives to main conclusions) is to 
get these new attitudes inculcated in all the producing 
divisions so that various and differing views are surfaced 
normally through the regular production mechanism.'' 
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the analysts and producing offices, rather than by another • 
entity or group organized and tasked specifically to 
prepare opposing views. This means that at each step 
along the way, drafters, branch and division chiefs, other 
offices, and colleagues in other agencies should continually 
question judgments. Clearly, the producers feel that, as 
professionals, taking account of differing views and con- 
tingencies is simply part and parcel of producing rounded, 
substantive papers. And normally this is sufficient— as 

the AD/DCI/NIO put it, "the Intelligence Community i s rig ht 
most of the time." 

Other practical reasons were advanced against 

institutionalizing a challenge mechanism, and they are set 
out. below. 

1) New Procedures. Under the new procedures • 
now m effect, a very considerable effort is made to build 
dissenting views into the text, and this goes some- distance 
to meet the problem. Interestingly, both a former DDI 

and Ambassador Komer regard this as a cosmetic touch (though 
both oppose institutionalizing the challenge mechanism) ; 

2) ^y£es_o^_Pa£er£. The AD/DCI/NIO stresses ' 
ohat a number of papers, such as the enormously complex 
and detailed NIE 11-3/8-74, "do not lend themselves to 
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be selected out for special treatment. But it would be 

very difficult to challenge the estimate as a whole, other 
papers where there is a deficiency of information or the 
information is ambiguous, could be challenged much easier. 

3) PCI Responsibility. The AD/DCr/KXO makes 
the point that the DCI is charged with presenting an 
estimative judgment to the President of the most likely 
developments in given situations, m this connection, 
he believes that having laid out our best judgment (with 
suitable space for dissent in the text) , it would be” terribly 
confusing" to readers to find another view set out at 
(as suggested in paragraph 4, TAB A). An NIO asked, "what 
can you do after the Devil's Advocate cites another position- 
s imply ask the policymaker to worry about it?,” "even though 
we have no basis for conceding” that the DA assessment is 
indeed the correct one. Still another NIO: the "Devil’s 
Advocate would quickly run out of steam with his analytical 
colleagues" since "all analysts work from the same jam pot ' 
of evidence the DA doesn’t have anything (other than his 
opiniorO to impress his analytical colleagues." 

4) Deadlines. The point is made that not ' 

infrequently important papers must be prepared for the 
NSC on very tight deadlines. One manager. says, "when you 

10 
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to attack your paper— you have to use every possible minute 
to get the paper produced." 

The concept of a Devil's Advocate is also criticized 

on grounds of artificiality, a former Assistant Deputy of 

O/NE claims that in his experience the "DA role drives 

an individual to take increasingly extreme positions, partly 

because he and everyone else knows that he is , role-playing'* ‘ 

and this contributes to an essentially artificial situation. ' 

A former DDI believes that, while a DA should not be 

institutionalized, the "ph ilosophy of a DA should be v 

established so that it permeates the working level s of 

intelligence production." A current manager is seeking 

to make progress in this direction by encouraging his analysts 

"if necessary to manufacture other alternatives to their 

main conclusions." But he too believes this is best accomplished 

by pushing this approach through the regular production 
mechanisms. * 

Interestingly, this manager thinks the culture in which 
the analyst works makes familiarity and ease in which differing 
views are surfaced hard to come by. He put it this way: ' 

fln analyst is flooded with information from many sources 
and then literally urged to make judgments and come to 
conclusions. Having done that he is forced to defend those - 
judgments up the line. Once he gets his views set, and 

11 - 
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difficult to^et the analyst to accept other differing 
views. 

Another knotty problem involves the best timing in 
using a challenge mechanism for best results, Some believe 
a DA could be useful before a paper is written, say in 
an "oral contributions" meeting. Others contend that the 
service would be chiefly useful -once a draft is prepared 
(this seems to make the most sense if the timing is suitable) . 
Still others would apply the challenge mechanism concurrently 

with the paper (perhaps as an annex as mentioned in the 

* 

memorandum at TAB A) or once a paper is produced, A 
Departmental Special Assistant for National Security suggests 
that the option to levy a Devil ’ s Advocate study on any 
given paper constitutes part of the pre’-USIB briefing process 
for the DCI . The Special Assistant believes the DCI is 
uniquely situated to determine whether a challenge to a 
paper should be instituted. But views are mixed; Ambassador 
Koraer beliqves this would be too late to be helpful. 

Behind all of the various doubts raised about the effec’- 
tiveness of a challenge mechanism. of any kind, is the very 
touchy problem of the controversy that is nearly certain 
to envelop any such body. The retired professionals make 
no bones about this as a fundamental consideration. One 

m 

commented that, whoever runs the NIOs would seek to "do in*' 

12 
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become highly controversial" and even a "high-powered" 

review board "probably can't be located in the bureaucracy 

Another thought it "might be done once,” as George Ball ' 

did on Vietnam m the State Department, but would not work - 
in general. ; 

Understandably, perhaps to avoid appearing negative, 
current managers did not volunteer comments on this point. 

One Bio who opines that he "likes the idea of a DA in certain 
instances," still believes it should not be institutionalized 
'■in the usual sense." He believes "that if it ware localized 
in, say, the Office of Political Pesearch of CIA, or on the ' 

IC Staff ' the people dol "Sr the job would quickly wear out 
their welcome; moreover, they would also tire of constantly 

acting as scolds to the Co^unity, a very unrewarding function 
The further point is made that "hostility brought down on 
these offices through the challenge function would make it 
harder for them to carry out their other missions. ' 

mile this consideration might be termed a mere "bureau- 
cratic problem, " in the sense of organizational positioning, 

“ ESSenCe “ that. It touches the matter ‘ 

of the size of an organization conducting the challenge (on 

this more later) and the crucial consideration of acceptability 
within the Community. An NIO said, "anyone acting as a Devil's 
Advocate would have to have the proper credentials as a know-' 
ledgeable individual in whatever area was under discussion." 

13 
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A manager claims that "if set apart from V ' regular working 

the attributes of a god," and wondered who could really 
meet the standards. Even with highly suitable credentials 
all who chose to comment agreed that controversy would, be 
endemic to the challenge mechanism. 

While the credentials of the "challengers" was cited as 
a critical factor, skepticism was still expressed about the 
chances that even a "high-powered” review group could be 
effective. In part, this concerns the comments on page. 10, 
"What should policymakers do? - worry?" It also relates to 
the difficulty in overturning the views of a large bureaucracy 
with officers steeped in the detail of the ‘daily traffic by 
a small group. 

The NIC staffer writes (after commenting on the people 
and resources of the current intelligence shops) "There is 
simply no way that a small group of indications specialists 
(read Devil's Advocates) can compete or really get an equal 
hearing when- they may be in disagreement." She also acknow- 
ledges, After all, the area specialists are not often very 
wrong and usually are able to offer impressive numbers of 
facts and persuasive arguments for their conclusions. Day 
in and day out, the odds are that the area specialists will 
be right" and "no group of this type (warning but equally 

« m- 
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of the current intelligence shops worldwide and keep a 
knowledgable watch on every potential adverse development . 11 
Ambassador Komar is more blunt: the people who follow the 
day-to-day traffic can "kill" any such review group, over- 
whelm it with data. And a former DD.I is equally clear, "A 
body or small group set off from the working level analysts 
would be overwhelmed by the amount of facts and details. that 
those working on a subject daily are able to bring to bear 
to support their case." And there is grave doubt expressed 
by most of the retired professionals that external consultants 
v/ould have sufficient familiarity with the world scene to 
be an effective counter. 

In a related view touching on the utility of a challenge 
mechanism, a"'former senior intelligence official sees some 
danger in the production of a dissenting view following an 
NIE. For example, after a Community view has been hammered 
out, a dissenting paper (apart from dissents registered in 
the estimate itself) would offer those pushing a "worst case" 
view a new opportunity to "merchandise their views to policy- 
makers.” The D/DCI/NIO sees this as "confusing" policymakers - 
An NIO thought "it would be a mistake to publish any sort, 
of Devil's Advocate paper." 

In considering the foregoing, an observer must conclude 

■* •- 

there are vastly different perceptions at work with respect 
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D/DCI/IC, who authored the paper at TAB A. For example, 
he views his actions over the years in repeatedly and 
consistently sounding a warning of potential Sino-Soviet 
warfare as a positive service to the policymaker. Fo matter 
that the judgment did not prove out (or at least has not in 
the period 1967-74). , 

f 

Others charged with providing estimative papers to the 

policymakers focus on the need essentially to provide correct. 

judgments (with suitable dissents and uncertainty made clear 

to the President and policymakers generally). These folk • 

» - 

see the emphasizing of the minority view as a confusing element 
if not an outright disservice to the policymaker. They 
consistently came back to the point — in the words of an NIO, 
"But the judgment on the Sino-Soviet struggle was flatly 
wrong.” Much thus depends upon your point of view: Should 
the DA sound the tocsin or is the game to estimate correctly 
(with due consideration for uncertainty) . 

Perhaps' ’a way out is suggested by a former O/FB official. 
He opines: "An estimate or substantive paper should come, 
down hard, as hard as the evidence permits, on a judgment, 
and it should be as pointed and precise as possible. But in 
those instances where the outcome on the other side of the 
majority position would be very serious to US interests, then, 
a "worst case analysis" should be undertaken. A kind of 

43 
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This would be a very thorough study of the' implication s 
of being wrong.” But he views this as very different from, 
and much more important than, an institutionalized challenge 
mechanism. A former DDI lends some support to this approach. 
He believes that if the concept of a’ challenge mechanism 
has any merit, "it is probably in those cases where the 
minority view occurrence , should it take place , would have 
very serious consequences for the U.S." In these circum- 
stances, he thinks the DCI might request that a post-NIE paper 
might be produced, though it should be a very closely held 

•u 

document. * ' 

It is apparent that among those polled, few envisage 
the successful institutionalization cf a challenge mechanism. 
It is equally clear that there would be rather considerable 
bureaucratic turbulence and resistance (possibly much of 
it subconscious) from other producing elements should a 
review entity be devised. . . 

III. POSSIBLE CHALLENGE MECHANISMS - SIZE AND LOCATION ' 

Nevertheless, the DCI must decide whether a challenge 
mechanism would suit him and his operating style. In part, 
it would depend upon his confidence in his producers. Is 
he confident that the working procedures in fact permit, 
indeed encourage, the surfacing of divergent judgments? 

17 
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t * lat ^Jggj-J-y has some clout worthwhile if it can also 

provide the DCX additional assurance that the best intelli- ' 
gence is being produced? 

If it is decided by Higher Authority that some sort 
of challenge mechanism is desired,, questions of organization 
and location, touched on previously, become immediate. 

Below are sketched out some of the options , in very general 
terms. While we have introduced some of the pros and cons, 
and qualifiers, with respect to these options, the following 
is not meant as an exhaustive treatment. Our main conclusion - 
is, perhaps predictably, in the spectrum of possibilities, 
the ones least likely to have a real impact on the substantive 
community are generally the ones most acceptable to the 
producing managers, that is, they are disposed to accept 
them and xvork with them. 

Considerations of size, location, and rank (of the office 
or individual) are closely intermingled. Possible challenge 
mechanisms include: . 

A single, very impressive individual, word'iLy , 
perhaps a scholar or well thought of retired 
Ambassador or public figure. He would be situated 
in the DCI’s immediate office and would enjoy very 

close working relations with him. Another proposal 

* . 

would envisage a slightly less formidable figure 
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Ambassador Komer and others think that it would not 
be feasible to place the function in the office of 
the D/DCI/NIO - Komer feels the individual would have 
to have at least equivalent rank with D/DCI/NIO. 

Three or four very impressive and knowledgeable 
figures— functioning as a Senior Review Board and 

attached to the O/DCI and with very close, continuing 
access to him. - 

An ad hoc "challenger." A different individual on 
each major paper perhaps selected by the NIO from ' / 

anywhere in the Intelligence Community. One suggestion 
is that this individual should be a high ranking 
substantive officer/ e.g., D/OCI or the Deputy DDI, 
pulled off his normal duties for a period of 2 - 3 
weeks to immerse himself in the subject. Clearly 
this latter proposal carries with it severe practical 
difficulties (as do all the rest) * 

The fcflO could conduct his own challenge though 
it is acknowledged to be difficult to, in effect, 
act as a DA against one's own paper. 

The Program Review Division (PRD) of the IC Staff 
This element already has a product review mission 
and has some substantive capability. (But see earlier 

m 

comments, page 13, which indicate this could hurt 
other IC Staff functions.) 7 
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Contains much of the former O/ME Staff - good 

analytic and critical staff. (Also see comments , 
page 13.) 

A new office— equal in size to the NIO structure. 
Ambassador Komer believes that to offset the other 
large analytic staffs, "any adversary group must 
be of the same size as the NIO set-up, including 
clerical assistance." He goes on to say; ,r if ft 
were not of adequate size, the DA would be working 
on Japan while some other crisis was emerging. The " 
DA would miss the new crisis and criticism would 
fall on him. Everyone would say, 'We have a DA, 
why wasn't he watching out for this crisis?'" 

Also, "any review group would need to have access to 
the vast amount of intelligence material available 
to the analysts," and this translates to a fairly 
large staff even to review only selected papers and 

issues. Comments on pages 14/ are pertinent. Probably 
located in the 0/DCI. 

Ad hoc consultants on given subjects. They would 
require access to all the intelligence and be asked ’ 
to prepare the challenge case. This would require 

a fairly leisurely production schedule, unless the 

* 

' 20 , ' 
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Was issued * These consultants could be responsible 
to the O/DCI, NIO, IC Staff or elsewhere. 

Panels of two to three consultants — experts in 
given areas, available to come to Washington on call , 
"almost as for jury duty." The author of this proposal 
admits to grave difficulties in achieving this kind of 
consultant constancy and response. Those who experienced 
the old O/NE consultant arrangement are skeptical over 
consultant arrangements generally.' a former DDI, still 
in the Washington area and immersed in a think-tank 
milieu does not feel he can stay current and doubts 

that others from academe can either. Organizational 
link as above. 

Another suggestion which is generally acceptable to all • 
would be the selection of a topic or two with an experimental 
DA. This has been done three times already ^ in one form or 
another. As noted in the case described in TAB. A, it was 
judged to have "helped to stimulate discussion." A Devil's 
Advocate view was also propounded in connection with NIAM 
11-9-74, soviet Detente Policy. The NIO concerned thought / 
the presentation by a knowledgeable PRD/IC Staff analyst 
had caused him "to look at a particular proposition more' 

closely, though it did not much change the NIAM." 

* 
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Intelligence Community's ability to monitor Soviet compliance 

with the SAL on offensive missiles, and lobbied for analysts 
to join him in a DA role, as a full-time occupation. 

Analysts at the meeting "objected that such a group would 
soon be discredited as a 'bunch of cranks." 

It is open to g^st ion whether additional experimen- 
tation would make the utility of a challenge mechanism more 
or less clear cut. in any event, many of the. problems 
involved m the institutionalization of challenge would 

simply be delayed until it became clear whether the OCX was 
going to take that vital step. ‘ ‘ - . 

Finally, whatever form the challenge mechanism took, 
the question of timing-when it want into action— would still 
be a very difficult one. it should probably evolve after 

a number of experiments. One point seems clear, that the 

challenge mechanism should probably be employed sparingly 
and only on very major issues. • . - 

In conclusion, as one of the pros said, the institution 
of a DA is not so important as the philosophy in producing 
substantive intelligence. Or as one s. Kent said many years 

ago (in reflecting on the substantive problem in strategic 
intelligence) : 


alr<n IJ h J ° nly answ f lies in picking a man who 
already knows a good deal about the substantive area 
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that he has ready access to every scrap of new- 
incoming evidence on it, access to everyone who 
knows about it, and freedom from other burdensome 
duties." 


•Se 


* 


* 
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INDIVIDUALS CONTACTED^ DURING THE STUDY 


retired professionals 

Ray Cline 


' CURRENT MA MGERS 

George Carver 
Ranh Knoche 


,NIOs 
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MEMORANDUM FOR: Director of Central Intelligence 

SUBJECT: The Devil’s Advocate 


1. On 13 December, as a result of a discussion I had with, 
the D/DCl/NIO, one of my. staff served as Devil’s Advocate (DA) 
during a Community-wide gathering of analysts on China. The 
DA argued that the likelihood of Soviet-initiated hostilities against 
China is greater than the Intelligence Community currently estimates. 
This is in line with the notion of establishing more effective challenge 

mechanisms in the production of intelligence. ' • 

* 

2. The DA argued along these lines: 

--The Intelligence Community has a tendency. to down- 
play the likelihood of hostilities — e. g. , prior to the most 
recent Arab-Israeli war. 

--The Soviets probably seriously considered initiating . . 
hostilities in 1969, and having done so once they might do 
so again, and decide differently. 



—The Intelligence Community’s predictions since 1969 - |i 

that the "Soviets have virtually completed their buildup near . . - j 

the Chinese border have been erroneous and misleading. ' :■ j 

Part of the problem is that the Community tends to look at 
evidence in a preconceived way. For instance, a new Soviet - A 

corps headquarters identified in January 1973 5.s viewed as * 

the tail end of a buildup rather than as the beginning of a new 
buildup — as a corps headquarters proved to be in November . 1 

1965; and a new division, overlooked when the September 1973 
NIE 31-13-73 was completed, is counted only reluctantly (it • - 
is said that the division equipment may only he there for { 

storage purposes). * * t 
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The Intelligence Community tends to equate the 

Soviet military planners’ views of the feasibility of a 
nuclear exchange with China with a U. S. military 
planner’s view of the U* S. -Soviet nuclear balance, 
which can be a misleading parallel. 

In Annex C of the September 1973 NIE, it is said 

that "the Soviets would calculate the chances of a Chinese 
retaliatory strike as slight" if the Soviets struck first 
with their nuclear weapons; yet the implications of this • 
judgment are not really applied to the rest of the NIE. 


— The Intelligence Community has a preconception 
that the Soviet leadership makes difficult foreign policy 
decisions on the basis of a moderate compromise, that 
is 9 the way Western cabinets tend to operate. But it • 
may be that when the Soviet leadership postpones decisions 
for a considerable period, the ultimate decision will be 
impulsive, incautious, interventionist. 


--There are three contingencies which could easily 
converge and increase the attractiveness to Moscow of . - 

military action against China: (1) the Chinese could draw- 
close to, but still lack both an effective tactical nuclear • . 

missile capability, and the capability of striking the 

Soviet heartland with strategic missiles; (2) the departure 

of Mao or Chou or (both could occur soon with one side in subsequent 

internal power struggle 'turning to an outside power, 

the USSR, for help; (3) the jSoviets could perceive that the 

U. S. is less able now to react to a Sino-Soviet war than 

it has been in the past or will be in the future. 


3. The DA believes that presentation of a "coherent” (not 
his term but that of one of the other participants) case helped to 
stimulate discussion. The question now is: how can DA roles 
be institutionalized? One approach would involve the designation, 
by the NIO in charge of drafting a major paper, of someone who 
would review previous papers on the subject in order to find 
loopholes. That is, even if the major judgments in the NIEs 
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Attached for your information are copies of 
a note Sam Wilson wants to send to theDCI 
and of my comments thereon, expressed in a 
note to Sam. The two annexes Sam refers to 
are rather lengthy and, hence, were not re- 
produced. They are, of course, available for 
your inspection should any of you care to 
peruse them. I would like to discuss this 
matter a bit at our session on 13 February. 
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